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Abstract 

Using the Internet to administer homework allows us to determine if students change their homework 
habits during a semester and if this change results in an improvement in grades. 


I. Introduction 



n introductory physics courses at Central Michigan University, the Computer-Assisted Personalized 
Approach (CAPA) has been used to distribute, grade and administer homework over the Internet. 1-3 In 
previous articles 4 ’ 6 we described how CAPA was used to directly measure when college physics students do 
their homework. These articles showed how the Internet was used to examine student study patterns and gender 
differences in homework 4 ; related homework behavior to performance in class 5 ; and identified collaborative learning and 
other successful strategies for on-line homework. 6 In this paper we examine changes in students’ homework behavior of 
over the course of a semester. In particular, we will look at changes following midterm grades. 


At Central Michigan University midterm grades are given in all 100-level and 200-level classes. One of the 
arguments given in support of issuing midterm grades is a belief that students not doing well in a class will change their 
ways in an attempt to improve their grades. This hypothesis will be tested for three college physics courses. First, the 
nature of CAPA data is described and general homework patterns are identified in relation to the structure of the course. 
A significant amount of attention is then given to the formulation and analysis of two homework behavior parameters, “P- 
DayMean” and “D-DaySpread” values, in relation to students’ performance. These variables will be used to quantify, 
respectively, the average day students do their homework and over how many days they spread out working on their 
assignments. 


II. Description Of The Course And Students 

The data collected in this study were taken from three introductory, algebra-based college physics courses taught 
in the spring of 1998, fall of 1998, and spring of 1999. Each course had enrollments of about 100 students. For each 
course, eleven CAPA problem sets were assigned and accounted for forty percent of each student’s grade. Every time a 
student submitted an answer to a problem a record of the submission was appended to the CAPA log file. The log files 
recorded the date and time of submission, which problem was attempted, and the result of the attempt. All students in 
each of the classes were required to submit answers to their assignments using CAPA and every student was usually given 
twenty chances to correctly answer each problem. Each time a student submitted an answer to a problem the student was 
identified, the time and date of submission were recorded, and their answer was registered as correct (Y), incorrect (N), 
wrong units (U), or wrong significant figures (S). Any submissions that included attempts at a multiple number of 
problems were dissected into separate submissions so that every revised submission contained an attempt on only one 
problem. Throughout this study, each of these revised submissions is referred to as a “hit.” 
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Table 1 displays a breakdown of the total hits for the Fall 98 class. The total number of attempts on all 
problems during the semester was 49,756 and there were, on average, 3.2 hits per problem for each student. Overall, 
seventy-five percent of all hits consisted of either correct or incorrect answers, while the remaining one-fourth were 
incorrect attempts due to wrong units or inaccuracy in significant digits. Table 1 also gives a general indication of the 
difficulty of each assignment in terms of the average number of hits per problem. For example, students appear to have 
had less difficulty with set 10 (2.4 hits per problem) and appear to have had more difficulty with set 5 (3.8 hits per 
problem). The hit summaries for the Spring 98 and Spring 99 semesters were similar; however, the students in those 
sections, who were in their second semester of physics, appear to have had less difficulty with significant digits. 


Table 1. Summary of all CAPA hits for fall 98. 

The l ast column lists the average number of attempts on a problem per student. 



# Probs 

"Y" Hits 

"N" Hits 

"U" Hits 

"S" Hits 

Total Hits 

HPP 

Set 2 

22 

1916 

3569 

186 

1390 

7061 

3.0 

Set 3 

14 

1299 

2979 

300 

1140 

5718 

3.8 

Set 4 

19 

1580 

3511 

262 

1227 

6580 

3.3 

Set 5 

13 

936 

1949 

426 

1262 

4573 

3.8 

Set 6 

19 

1450 

2537 

355 

1296 

5638 

3.3 

Set 7 

12 

1010 

2297 

80 

760 

4147 

3.6 

Set 8 

19 

1582 

1976 

254 

1261 

5073 

3.0 

Set 9 

14 

1219 

1063 

110 

644 

3036 

2.6 

Set 10 

16 

1394 

1650 

139 

407 

3590 

2.4 

Set 11 

16 

1353 

1869 

181 

937 

4340 

3.0 

Total 

164 

13739 

23400 

2293 

10324 

49756 

3.2 


28% 

47% 

5% 

21% 



While a more detailed breakdown of the hits can offer insight into where students may be having difficulty, the 
primary focus of this study was to use CAPA data to assist in determining students’ homework behavior. In Figure 1, the 
total hits for each class are distributed by day of the week, revealing an overall picture of when students did their work. 
The reader should be reminded here about the structure of the course. Assignments were always due on Sunday at 3:00 
AM, and were handed out either on the previous Monday (odd numbered sets) or ten days prior (even numbered sets). 
Each day in this study is defined as a twenty-four hour period beginning at 5:00 AM. For even numbered sets, all of the 
hits registered prior to Monday are grouped into a category labeled “Prior”. 

III. Homework Behavior And Performance 

The primary interest of this paper is to examine homework data in relation to how students performed in the 
course and to determine if midterm grades have an influence on either. This section introduces how the hit data can be 
used to formulate various types of values that describe when students did homework. 

Figure 2 displays the normalized hit distributions for five groups that are based on final grade. The 1st quarter 
group consists of the top twenty-five percent of the students who completed the class, while the bottom twenty-five 
percent constitutes the 4th quarter group. The seventeen students who, at some point in the semester, withdrew from the 
class make up the fifth group labeled “W.” Figure 2 clearly illustrates differences in the hit distributions between groups. 
Students in the top half of the class submitted seventy-five percent of their total hits by Thursday. Forty-one percent of 
the hits from the students in the bottom quarter occurred on Saturday, and students who ended up withdrawing did nearly 
sixty percent of their work on the last day. 
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(a) Spring 98 



Day of Week 

(b) Fall 98 



Day of Week 


(c) Spring 99 




Day of Week 

Figure 1. Total hits distributions by day of the week. 
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60% 



a 1st Qtr 

■ 2nd Qtr 
□ 3rd Qtr 

■ 4th Qtr 
mws 


Day of Week 


Figure 2. Fall 98 normalized hits distributions based on final grades. The top 25% of the class constitutes 
the 1 st Quarter Group; the 5 th group labeled ‘W’ is comprised of the 17 students who withdrew from the 
course. 


In order to analyze the differences in homework behavior between groups, it is necessary to quantify the hit 
distributions. There are several ways to use the hit data to develop values that describe an individual student’s homework 
behavior. For example, the “DayMean” value depicts the mean day for which a student submitted answers to CAPA. A 
student’s composite DayMean value (//,„,) for the semester is computed by averaging the day values (see Table 2) over all 
of the student’s hits ( n lo ,) for the entire semester: 

n tot 

V d 

L—t n 

li — 

r^tot 

n„„ 


Table 2. Day values assigned to individual hits. 


Dav of Week 

Dav Value (d) 

Prior 

1 

Monday 

2 

Tuesday 

3 

Wednesday 

4 

Thursday 

5 

Friday 

6 

Saturday 

7 
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Likewise, a student’s DayMean value for a particular assignment is computed by averaging the day values over 
all hits submitted for the particular assignment. 

An underlying assumption in the use of the DayMean value as an estimate of when a student did homework is 
that some degree of work was associated with each hit (i.e. some computation, re-computation, and /or some thought 
process involved with solving a physics problem). However, if one were concerned with a weighting effect due to 
differences in the hits per problem ratio, then an alternate method would be to use “p-hits”. A single p-hit occurred when 
a student attempted a particular problem, regardless of the number of attempts associated with the problem. There was 
one p-hit possible for each day of the week for each problem. For example, if a student incorrectly answered problem #4 
two times on Wednesday, and it took five attempts on Thursday to correctly answer the problem, then one p-hit (for 
problem #4) was recorded for Wednesday and another was recorded for Thursday. Using p-hits, the “P-DayMean” value 
is defined as 


Pp 



U e 


where n p is the total number of p-hits for a student. 

Figure 3 displays the average P-DayMean value for each of the five groups based on final grades. An analysis 
of variances with various post-hoc tests shows that significant differences (p<.05) exist between every other group. For 
example. The 1 st Quarter group achieved significantly lower |i p values than did the bottom half of the class. Likewise, 
students in the bottom 25% of the class achieved significantly higher p p values than did students in the top half of the 
class. There was also a marked difference when students in the top half of the class did their homework and when 
students who withdrew from the course did their homework. 


The P-DayMean value can be regarded as a statistic of central tendency. An additional measure of when a 
student did homework would be the extent to which he or she did homework on a daily basis. Unlike with most statistics 
of variability, the standard deviation cannot be used as a measure, since the measure should not concern itself with which 
day(s) homework was done. Instead, the “D-DaySpread” value for a particular assignment is simply taken as the number 
of days for which CAPA activity was recorded. The composite D-DaySpread value (ad) for the entire semester is taken 
as the average of the spread values for each assignment. A composite D-DaySpread value a d = 2.0, for example, 
indicates that a student had, on average, worked on homework two days for each assignment, regardless of which days of 
the week. Figure 4(a) displays the distribution of a d values for the Fall 98 class. As the graph illustrates, a majority of 
the students had spent two days on their homework, while less than a third had spent three or more days on their 
assignments. 

The relationship between a d and final grade for the Fall 98 class (Fig. 4(b)) was observed to be statistically 
significant and nearly as strong as the relationship between p p and final grade. Students in the top 25% of the class 
clearly spent more days doing homework (c d =2.6) than did students in the bottom fourth of the class (a d =1.7). 

The reader should be reminded here that 60% of a student’s final grade was based on five exam scores, while 
40% was based on homework scores. While it has been shown that moderate relationships exist between final grades and 
when students did homework, differences were observed with respect to performance on homework and exams. For the 
Fall 98 class, P-DayMean was strongly correlated with homework scores (r=-.67). Although students with lower |i p 
values tended to do better on exams as well, the relationship was not statistically significant (when all students were 
included). On the other hand, D-DaySpread was correlated with both homework scores (r=.50) and exam scores (r=.20). 
A moderate relationship was observed between homework scores and how students did on exams. Table 3 summarizes 
the Fall 98 correlation data. 
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1 st Qtr 2nd Qtr 3rd Qtr 4th Qtr W 

Groups Based on Final Grades 


Figure 3. Average P-DayMean values for groups based on final grades. Included are 95% confidence 
intervals. Significant differences (p<.05) exist between every other group. 


(a) D-DaySpread Distribution 



1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 

<T d 


The accuracy of the relationships observed above are of course dependent on how well the P-DayMean and D- 
DaySpread values reflect when individual students actually did their homework. Self-report data indicated that some 
homework activity occurred on days for which no CAPA activity was recorded. A survey was used to identify students 
who were likely to exhibit this sort of home work behavior. There were fifteen students who reported having worked most 
of their problems before logging onto CAPA, tended not to be near a computer when they did their work, and tended to 
submit most of their problems all at once. Exclusion of these fifteen students with suspect p p and a d values results in 
relationships with improved strengths and significance (shown in parentheses in Table 3). 
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1 st Qtr 2nd Qtr 3rd Qtr 4th Qtr W 

Groups Based on Final Grades 


N* 

98 

Pearson Correlation 

.41 

Sig. (1-tailed) 

.001 


Figure 4. D-DaySpread data for Fall 98. (a) Distribution of D-DaySpread values for the Fall 98 class, (b) Results 
from an ANOVA indicate that students who did better spent more days on their assignments. 


Table 3. Pearson Correlation data for Fall 98. Exclusion of 15 students with suspect p p 
and CT d values resulted in more accurate relationships with improved strengths and 
significance (shown in parentheses). 



Final Grade 

Homework 

Exams 

P-DayMean (p p ) 

-.50*’ 

(-.55”) 

-.67’* 

(-69”) 

-.14 

(-21*) 

D-DaySpread (o d ) 

.41” 

(.42") 

.50" 

(.46**) 

.20* 

(.25*) 

Homework 



.33” 


**p<.01 * p<.05 


* 3 outliers not included. 
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IV. The Changes In Behavior 

The analyses conducted thus far have shown significant relationships between overall performance and when 
students did homework. These analyses have been conducted using the composite p p , ct d values, which give general 
indications of when students did homework over the entire semester. Although the state of most homework behaviors 
could not be determined for any particular point within the semester, it was possible to evaluate changes in when students 
did homework by analyzing P-DayMean and D-DaySpread values for individual assignments. As an example, the 
general trend for the Fall 98 class was a decrease in P-DayMean values as students progressed through eleven 
assignments. As Figure 5 illustrates, performance on assignments tended to increase accordingly, and the correlation 
between the two variables was determined to be significantly high. 



Cl 

> 

< 


a P-DayMean • HW Score 


Figure 5. Average score on assignments vs. average P-DayMean values. The graph clearly indicates 
that, as the semester progressed, students did their work earlier in the week and homework scores 
increased accordingly. The Pearson correlation between the 2 variables was r=.77. 


It is conceivable that a student’s midterm grade could have had a significant impact on his/her outlook on course 
standing, and therefore homework behavior as well. An analysis was conducted to determine the extent to which both 
performance and when students did homework changed between the first and second halves of the semester. The results 
of the analysis are graphically summarized for all three sections in Figure 6. The left column in Figure 6 displays plots of 
changes in performance on assignments versus changes in P-DayMean; the column on the right displays changes in 
performance on exams versus changes in D-DaySpread. 

Although regression analysis indicates that each of the six sets of data constitutes a significant relationship, more 
useful information can be extracted from a visual interpretation of the plots. It is clear that a majority of the students in 
the Fall 98 class (Fig. 6(a)) exhibited better homework behavior after the midterm, as indicated by the changes in P- 
DayMean and D-DaySpread values. The plots for the Fall 98 section illustrate a significant improvement in 

Homework scores for those students who did homework earlier in the week than they had before the midterm, 
and also indicate an improvement in exam scores for students who spent more days per week after the midterm. 
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(b) Spring 98 
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(c) Spring 99 
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Figure 6. Changes in performance vs. changes behavior. The left column displays changes in normalized 
homework scores vs. changes in P-DayMean values. The changes are the differences between the first and second 
halves of the semester. Data containing little or no change in behavior is excluded from the plots. The colunm on 
the right displays plots of changes in studentized exam scores vs. changes in D-DaySpread values. All six sets of 
data constitute significant relationships (p<.05). 
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The data for the spring sections show similar relationships between change in performance and change in when 
students did homework. However, the plots for these classes also show that the change in behavior tended to be opposite 
of that for the fall semester sUidents. Many of the Spring 98 students achieved lower D-DaySpread values after the 
midterm and did not do as well on exams, as was the case for many students in the Spring 99 class. In addition, the 
homework plots for the spring sections indicate that after the midterm a majority of the students tended to submit their 
homework later in the week, and tended to achieve lower scores on their assignments. 

V. Conclusions 

Significant relationships fp < .001) were found when homework behavior and achievement in physics were 
analyzed with students’ GPA data. Not only were Day-Mean and Day-Spread values correlated to final grades in physics, 
they were also correlated (r = .5) to cumulative GPA prior to entering the course. More interestingly, homework 
behavior and physics grades were significantly correlated (r = .4) to an adjusted semester GPA that had physics grades 
factored out. This immediately suggests a possible relationship between homework behavior in physics and homework 
behavior in other courses. 

Isolation of the “P-DayMean” and “D-Day Spread’’ parameters before and after the midterm period also yielded 
interesting results. Generally, students who had exhibited better homework behavior after the midterm achieved better 
scores in relation to their performance before the midterm period. While this type of relationship was consistent among 
all of the sections, differences were observed with respect to which direction sUidents actually changed. For example, 
during the fall semester many students had worked on the homework earlier in the week after the midterm, and achieved 
higher scores on assignments. During the spring semesters, on the other hand, a majority of the students spent fewer days 
per week on assignments after the midterm; this change in behavior was accompanied by a drop in exam scores. 

One is tempted to blame this pattern on “spring fever.” (This is an issue we leave to the social science 
researchers.) A more important interpretation of the data involving changes in behavior, however, is that it substantiates 
the results mentioned in this and previous studies, 4 ' 5 ' 6 which are based on the students’ composite li p and g () values. In 
other words, the relationship between performance and homework behavior may actually be stronger than what has been 
presented because the composite values tend to “average out” the observed relationships associated with changes in 
behavior. 
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